Tech Arsenal 1

home *** CD-ROM | disk | FTP | other *** search

/ Tech Arsenal 1 / Tech Arsenal (Arsenal Computer).ISO / tek-04 / adacplus.zip / ADACPLUS.HLP < prev

Wrap

Text File | 1991-12-02 | 24KB | 429 lines

OVERVIEW OF U.S. AIR FORCE REPORT Ada AND C++: A BUSINESS CASE ANALYSIS Form G75-1191 AdaCPLUS.HLP Ada Information Clearinghouse, 1-800-AdaIC-11, 1-703-685-1477 On July 9, 1991, the Air Force released to the public a report of a business case they conducted to determine under what circumstances a waiver to the DoD Ada requirement might be warranted for use of C++, particularly in DoD's Corporate Information Management (CIM) program. The report is titled, "Ada and C++: A Business Case Analysis." Since its release the report has received a great deal of publicity in various newspapers and journals. The information that follows was excerpted from remarks made by Mr. Lloyd K. Mosemann, II Deputy Assistant Secretary of the Air Force (Communications, Computers, and Logistics), at a press conference held July 9, 1991. I. Introduction There has never been any intention to question DoD's commitment to Ada, but only to identify when waivers for C++ might be warranted. This business case will support the development of DoD programming language policy for information systems and C3 systems. I might say at the outset that language comparison is not merely a scientific issue: it evokes strong emotions as well, in that to a certain extent people adopt "favorite" languages for reasons other than purely dispassionate analysis, much as one might not be able to explain why he/she roots for the Chicago Cubs or drinks Coke rather than Pepsi. The task is also rendered difficult because there are yet no well-established and standard methods for conducting such comparisons. For these reasons, we endeavored to make our study as quantitative as possible, asking the best experts we could find to use a variety of methods that have historically been used for business analysis in such tasks. We felt that by using a variety of methods and comparing their results, we would avoid the skewing that might result from the sole use of a single method. In our business case, therefore, several different approaches were undertaken to identify, from a business perspective, when the life cycle cost effectiveness of C++ might be greater than that of Ada. -- The first, conducted by the Institute for Defense Analyses (IDA), examined quantitatively the availability of tools and training for the two languages. -- The second, conducted by the Software Engineering Institute, applied to this problem a quantitative language selection methodology developed by IBM for the Federal Aviation Administration (FAA). -- The third, conducted by CTA, Inc., analyzed cost and cost analysis). -- And the fourth, conducted by the TRW Corporation, applied a standard cost model in depth to both languages for a typical information systems/C3 project (micro analysis). -- In addition, the Naval Postgraduate School (NPS) was asked to address the overall policy issue of Ada, particularly in the context of emerging fourth-generation language (4GL) software technology. Each of the substudies reached the same conclusion: there are no compelling reasons to waive the Ada requirement to use C++. The business case analysis was directed at information systems and C3 systems. However, there is no reason to believe the results would differ for computer programs embedded in weapons systems. Let me now summarize for you the salient quantitative results of each study, and I think you will understand more fully how we arrived at our conclusion. II. Substudy Results. A. Tools, Environments, and Training: IDA Substudy. The Institute for Defense Analyses (IDA) collected and analyzed information on the market availability of commercial- off-the-shelf products available from U.S. sources for Ada and C++ compilers, tools, education, and training. The study provided a large quantity of demographic data. For example, there are 28 companies located in the U.S. that have Ada compilers with currently validated status; 18 vendors offer C++ compilers. The Ada compiler vendors are more likely to have been in business five years or more. Ada "validation" is more rigorous than that of other high order languages: only Ada is monitored and approved for conformity to a standard, without supersets or subsets, by a government-controlled process. By contrast, no validation or even a standard of any kind exists for C++, although a standard by 1994 is expected. Both languages are supported on PCs and workstations. Ada is also supported on mainframes. Ada, but not C++, has cross compilation systems. Ada is supported with program engineering tools. Compiler vendors provide a rich set. Code generators exist for Ada but none so far for C++. There is considerable variability among C++ products in language features supported and libraries provided. Ada is taught in 43 states at 223 universities and 13 DoD installations. C++ is taught in four states at four universities and no DoD installations. There are more Ada than C++ courses available. The cost of training is about equal, but Ada course variety is wider. B. Faceted IBM Language Selection Methodology: SEI Substudy. The Federal Aviation Administration (FAA) contracted with IBM in the mid-1980s to evaluate high order languages for use on its Advanced Automation System (AAS) Program. In response, IBM developed a formal, quantitative faceted methodology comparing 48 language features (criteria) in six categories. This IBM study concluded that use of Ada was "in the ultimate best interest of the AAS program and its goals, and that warrants coping with the temporary risks/problems that loom large in the near term in order to reap the significant benefits/payoffs over the long term." Using this same methodology for each of the 48 criteria, the Software Engineering Institute (SEI) evaluated Ada and C++ for application in information systems/C3 systems. The original FAA weighted scores for the six criteria categories were as shown in this matrix: Category Max. Ada C Pascal JOVIAL FORTRAN Score Capability 16.7 6.1 9.6 10.4 7.6 3.9 Efficiency 16.4 8.0 11.8 10.8 11.0 11.1 Availability/ 22.6 21.5 11.6 14.5 15.6 10.3 Reliability Maintainability/17.4 14.0 10.2 12.2 6.8 8.3 Extensibility Lifecycle cost 11.3 8.2 7.4 7.8 4.9 5.2 Risk 15.6 8.8 8.9 7.6 9.6 8.2 Total 100.0 76.6 59.5 63.3 55.5 47.0 The 1991 weighted scores for the six criteria categories were: Category Max. Score Ada C++ Capability 16.7 15.3 11.3 Efficiency 16.4 10.7 10.9 Availability/Reliability 22.6 19.1 12.6 Maintainability/Extensibility17.413.6 11.4 Lifecycle cost 11.3 8.4 8.0 Risk 15.6 11.7 9.8 Total 100.0 78.8 63.9 In 1985 Ada was considered considerably more capable than C. Today, the SEI study found there is still a significant difference between Ada and C++, C's successor. The relative efficiency of Ada has improved markedly; Ada still scores significantly higher in availability/reliability; the Ada advantage in maintainability/extensibility persists; and from a position of parity in 1985, Ada has attained in 1991 a significant advantage over C++ in lowered risk. An attachment lists numerous major Ada information systems/C3 systems. It is not widely appreciated that such extensive use is now being made of Ada: in fact, the Ada 9X Project Office reports that the U.S. Ada market, excluding training, services, and government research/development, now exceeds $1 billion. C. Macro Cost Analysis: CTA Substudy. CTA compiled and compared available productivity and cost data of Ada and C++. Much of their data comes from Reifer Consultants' extensive database, one of the best, largest, and most current programming language cost databases now available. Average productivity across the four domains for which data exists (environment/tools, telecommunications, test (with simulators) and other) for both Ada and C++ projects is shown in this matrix. Note the productivity advantage for Ada: Productivity Number of (SLOC/MM) Data Points * Norm (all languages) 183 543 * Average (Ada) 210 153 * Average (C++) 187 23 * First project (Ada) 152 38 * First project (C++) 161 7 The C++ project data reflected information on 23 projects taken from seven firms who had been using C++, Unix, and object-oriented techniques for over 2 years. All projects were new developments. Application size ranged from 25 to 500 KSLOCs (thousand source lines of code). Average size was about 100 KSLOC. The average costs across the four domains for both Ada and C++ projects are shown in this matrix. Cost Number of ($/SLOC) Data Points * Cost (all languages)$ 70 543 * Average (Ada) 65 153 * Average (C++) 55 23 Typically, the Ada developments were performed in accordance with military standards and incorporated formal reviews, additional documentation, and additional engineering support activities such as formal quality assurance (QA) and configuration management (CM). Most C++ projects are commercial and do not extensively incorporate such activities. Additionally, on such projects developers are typically intimately involved with users, resulting in considerably less requirements engineering effort. Consequently, applications on which C++ is used are inherently less costly, so that the reported productivity rates are favorably skewed toward C++. The average error rates across the four domains for both Ada and C++ projects were: Integration FQT Number of Error Rates Error Rates Data (Errors/KSLOC)(Errors/KSLOC) Points * Norm (all languages)33 3 543 * Average (Ada) 24 1 153 * Average (C++) 31 3 23 The integration error rates include all errors caught in test from start of integration testing until completion of software Formal Qualification Test (FQT). The FQT error rate includes only those errors found during the FQT process. A so-called "transition state analysis" performed by Reifer's group indicates that 26 of the 38 firms within the Ada database had successfully made the changeover to effective use of Ada, while none of the 7 firms in the C++ database had made the transition. Also, none of the 7 firms were fully using C++'s inheritance and other advanced features. The standardization maturity of Ada was found by the CTA to be particularly important. While Ada has a firm and well policed standard, allowing neither supersets nor subsets, it will be years before a stable C++ language specification is established. New features are being considered for the latest standard C++ release. Vendors are likely to offer their own enhanced versions of C++ compilers and CASE tools, complicating portability and reuse. Finally, the original arguments for establishing a single programming language for military applications were found to remain. Common training, tools, understanding, and standards simplify acquisition, support, and maintenance. The study concluded that after maturing for a decade, Ada's benefits have been proven for all application classes. Ada projects have reported 15% higher productivity with increased quality and double the average size. Normalizing these data to comparable size projects would result in an expected Ada productivity advantage of about 35%. Ada should be the near term language of choice. C++, the study felt, still needs significant maturing before it is a low risk solution for a large DoD application. D. Micro Cost Analysis: TRW Substudy. TRW performed a tradeoff analysis that generalized recent corporate cost analyses on a typical real-world information systems/C3 systems project. Their study defined a set of maximally independent criteria, judged each language with respect to those criteria, and then translated those judgments into cost impacts to emphasize the importance of each criterion from a lifecycle cost perspective. Results were translated into perturbations of Boehm's Ada COCOMO cost model. Rankings of the two languages based on this analysis are shown in this matrix (0 = no support; 5 = excellent support), followed by a total score, a weighted sum of the rankings based on weights determined by an expert panel: Ada C++ Reliable S/W Engineering 4.5 3.2 Maintainable S/W Engineering 4.4 3.2 Reusable S/W Engineering 4.1 3.8 Realtime S/W Engineering 4.1 2.8 Portable S/W Engineering 3.6 2.9 Runtime Performance 3.0 3.6 Compile-Time Performance 2.3 3.1 Multilingual Support 3.1 2.4 OOD/Abstraction Support 3.9 4.6 Program Support Environment 4.1 2.1 Readability 4.4 2.9 Writeability 3.4 3.5 Large Scale S/W Engineering 4.9 3.3 COTS S/W Integration 2.8 3.6 Precedent Experience 3.6 1.5 Popularity 2.8 4.0 Existing Skill Base 3.0 1.8 Acceptance 2.5 3.3 TOTAL SCORE FOR MGT INFO SYSTEMS 1631 1324 (Ada score is 23% higher) TOTAL SCORE FOR C3 SYSTEMS 1738 1401 (Ada score is 24% higher) The study concluded that both Ada and C++ represent improved vehicles for software engineering of higher quality products. Currently, C++ was estimated to be approximately 3 years behind Ada in its maturity and tool support. The case study used in this report (the Command Center Processing and Display System-- Replacement) demonstrated development cost advantages for Ada on the order of 35% and maintenance cost advantages for Ada on the order of 70% under today's technologies. In the far term (1994+), the study felt, this Ada advantage might erode to approximately a 10% advantage in development costs and 30% in maintenance costs for a typical development intensive system. The study listed the primary strengths of Ada as its support for realtime domains and large scale program development. Its primary weaknesses are its compile-time and runtime efficiency. The primary strengths of C++ listed were its support for better object oriented design, support for COTS integration, and its compile-time and runtime efficiency. Its main weaknesses were identified as its support for reliability and large scale program development. In general, the study felt Ada's weaknesses to be solved by ever-increasing hardware performance and compiler technology advancement. C++ weaknesses, on the other hand, remain to be solved by advances in its support environment. E. Ada Policy Issues: NPS Study. Concurrently with the preparation of this Ada and C++ Business Case Analysis, the Naval Postgraduate School (NPS) reported on policy issues on the use of Ada for Management Information Systems. Their report, an analysis of the need to see Ada in a total and evolving context, is an important vision statement leading from Ada as the primary third-generation language (3GL) to its conception as the basis for evolving to higher levels of productivity in so-called 3 1/2 GL and 4GL environments. Rather than concentrating on programming language selection, the NPS report focuses on and argues for needed advances in software development technology. In particular, the Report contends, while traditional factors such as programming language selection, better training, and computer-assisted software engineering (CASE) tools can enhance productivity modestly, a fundamental change in the software development paradigm will be necessary to achieve an order of magnitude gain. Such a gain is possible through use of 4GLs, languages that will ultimately enable the developer to define the complete design of an application entirely in the 4GL's own high-level specification language. The specification is then translated automatically by the 4GL into an executable program. When accompanied by a productive development environment, an evolutionary implementation methodology, and well trained development teams, the report asserts, 4GLs can provide a tenfold gain in productivity. An intermediate step cited by the report in the movement to 4GLs is 3 1/2 GL programming, a term referring to the extensive use of CASE tools coupled with a high level of code reuse. The 3 1/2 GL approach requires a strong commitment to codifying and accrediting code modules, to the point where it becomes easier and more desirable to reuse code than to rewrite it. Although experience with 4GLs has not yet been extensive (with existing experience limited largely to specific functional domains such as financial management and transaction processing), 4GLs are attractive for several reasons. One is their robustness under change: changes made to the application, for whatever reason, are made at the specification level and then re- translated automatically into executable code. Another is the facility with which they can be integrated into tightly knit and full-featured development environments. For these reasons, the report strongly recommends that the DoD discourage use of traditional 3GL programming and take bold steps to incorporate the 4GL paradigm. Finally, the report recommends that, given the importance of Ada to DoD software, greater effort and funding should be provided for the key Ada initiatives: the Ada Technology Improvement Program, Ada 9X, and Ada education initiatives. Two issues on 3 1/2 GLs and 4GLs related to this business case were outside the scope of the NPS report. The first of these is that, for the foreseeable future, state-of-the-art limitations will probably keep 4GLs from generating more than half the total code required by most applications. In such cases, where a substantial amount of 3GL programming will be required to complete application development, use of a 3 1/2 GL approach, rather than a 4GL approach, is preferable. Another issue outside the scope of the NPS Report was the evaluation of the relative merits of Ada and C++ as target (output) languages for 4GL application generators. However, as section V.C of the NPS report points out, a "standard, stable target language portable to a variety of hardware platforms" with good software reuse and interface definition capabilities is appealing. Although more study of the characteristics desired in 4GL target languages is warranted, the SEI and TRW substudies suggest no particular advantage of C++ over Ada in software reuse and interface definition, so there appears no reason to waive DoD's Ada requirement in favor of C++ as a target language for 4GLs. 3. Conclusions. All four substudies which specifically compared Ada and C++ (IDA, SEI, CTA, TRW) report a significant near term Ada advantage over C++ for all categories of systems. This advantage could be eroded as C++ and its supporting environments mature over the next few years. On the other hand, as aggressive overseas Ada initiatives stimulate even wider domestic Ada interest, as Ada tools/environments further mature, and when the Ada update (Ada 9X) is complete, the balance could tip further in Ada's favor. Adding to the case for Ada is the fact that the Ada scoring so well in the business case was Ada's 1983 version, MIL-STD-1815A. Just as C++ incorporates into C certain software engineering concepts already in Ada (e.g., modularity, strong typing, specification of interfaces), so an Ada update now underway will bring into Ada selected features now included in C++. This update, known as the Ada 9X Project, is targeted for completion in 1993. The product of extensive community involvement (including the C3 and MIS communities), Ada 9X will bring to Ada such improvements as decimal arithmetic, international character sets, improved input/output, support for calls between Ada and other languages, further representation specifications, and inheritance/polymorphism (popular features of C++). The Ada 9X Project Office lists one of the goals of Ada 9X as "to provide all the flexibility of C++ with the safety, reliability, and understandability of Ada 83." At the same time, Ada 9X has been designed so that neither existing Ada benefits nor performance will be lost. For example, Ada 9X inheritance will be controlled so as not to reduce lifecycle supportability. Some have criticized OOP features such as inheritance as potentially dangerous to DOD software mission goals (such as safety, reliability, and dependability). Bjarne Stroustrup himself, the originator of C++, has been quoted as follows: "C makes it easy for you to shoot yourself in the foot. C++ makes that harder, but when you do, it blows away your whole leg." In summary, it is not possible to make a credible case for the existence of classes of "more cost effective" C++ systems compared to Ada. Business cost effectiveness data collected for this study are typified by the TRW study's conclusion that Ada provides development cost advantages on the order of 35% and maintenance cost advantages on the order of 70%. In terms of full lifecycle costs, it will be some time before data exists which could justify a cost savings for C++. Today, there is limited lifecycle data available for Ada and almost none for C++. For the foreseeable future, then, this business case shows that there are more than enough reasons for the DoD to stick firmly with Ada, both for all high order language (3GL and 3 1/2 GL) development and for exclusive use as a target language of 4GL application generators in the large class of applications for which 3GL code must supplement generated code. "Ada and C++: A Business Case Analysis" is available in hard copy form from Software Technology Support Center, OOALC/TISAC, Attn: Gary Peterson, Hill AFB, Utah 84056, DSN 458-7703, 801/777-7703. It is available in electronic form in the ada-info directory on the AJPO machine on the Defense Data Network and on the AdaIC electronic bulletin board (703) 614-0215 or (301) 459-3865.